OOCQMEIIT BESDMB 



BD 089 77,1 



IB 000 485 



AOTHOB 
tITLE 



Lucasr Henry C, Jr* 

Systeis Quality, Ojser Reactions, and the Ose of 
Inforaation Syst^is. 

Stanford Onlv.f ballf. Graduate School of 

Business. 

SO-GSB-RP-193 

Jan 74 

20p. J Research Paper Munber 193 



IHSTITOTIOM 



BEPORT »0 
PUB DATE 
itOTE 



EDRS PBICE 
DBSCfilPTORS 



MP-$0.75 HC-$1.50 PLUS POSTAGE i 
AdilnlstratlVe Personnel; Attitudes! ♦Co«putersii' Data 
Processing; Evaluation; Infomation Procesising;, 
Inforiatlon Services; ♦Inforiatlon systens; 
Inforiation Otilizatlon; Onlversity Adalnlstratlon; 
♦Ose Studies 



ABSTRACT 




batch process and online coiputet \systeiis at a lajor university, 
i^espon^ed to a questionnaire designed to discover attitudes toward 
the quality of the system and services, the users* evaluation of the 
sdrvic«, and the relationship of these attitudes with voluntary use 
of\the systen. Theise attitude, evaluation, &nd use factors were 
consldei^ed in relation to the actual quality of the systea as rated 
indeDend<antly by the inforiation services departaent stiaff. Data wisre 
coll^ctied on input-output quality; adequacy of training; nanagQaent 
suppoijt; \guallty and kinds of reports generated; Input error ratings; 
usefulness; accuracy; tiieliness, flexibility; conpleteness; and 
pressur^ to reprogran the syste*. Step-wise lultiple regrjession was 
used to analyze the data, which did show an association between 
actual qua^ty of systems, and quality as perceived by users, with 
favorable tiser attitude. These favorable attitudes and evaluations 
were also positively associated with systea usage. Further, increased 
voluntary use aay be predicted by favorable attitudes, which provides 
good reasons for inforaation services to develop high quality systeas 
and favorable user attitudes. (LS) 
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SYSTEMS QUALITV, USER REACTION AND THE USB OP INFORMATION SYSTEMS 

llttRODUCTION ' ^ 

In two past studies the quality of computer systems as perceived by 

users was found to be^trongly associated with favorable user attitudes , 

toward the infotmatioX services department staff and with ratings of 

computer potential (4|5]« In these stiidiesi perceptions of systems quality 

and user reactions were all measured by the same data source, a user ques* 

ttonnalre. Is this association between attitudes and systems quality due 

to ^ "halo effect" from the questionnaire or do other independent measures 

of system quality have a positive association with u^er attitudes? Another 

question of Interest is whether the use of an information system can be 

/predicted by favorable user attitudes (usage measures were not included in 

, the earlier studies {4,51). ' \ 

' The purpose of this paper is to present the results of a study of ^ 

user reaqtlons, the use of information systems and an independent rating 

of systems quality* The paper also describes the factors associated with 

the use of an online^^syi^tem which has encountered a number of problems « 

Speciticallyi the^reseavch was formulated to test the following hypotheses: 

Hypothesis I: Favorable user attitudes are associated* with favorable 

user perceptiops of systems quality and Independent 
^ . ratings of systems quality. 

Hypothesis 2: The use of an information system is positively aseo- 
elated with f/^vorable user attitudesi favorable user 
perceptions of systems and favorable Independent 
ratings of systems quality* 

Issues related to quality of service are a major cause of the Interaction 
bclweon users and iUv Information services department staff. If this Intor- 
' action is favorable it HhoulJ result in the formation of positive attltiuU>M* 
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If the Jditei'actlon occutd under adverse clrcumatance» poor attitudes m^y \^ 
be the result. Users are expected to have fairly accurate perceptions of 
the quality of service even allowing for perceptual distortions created by 
unrepresentative inci4ents or problems. That l8» user evaluation of sys- ^ 
terns should be consistent with Independent raj^ngs of the <tuality of 
Inforiaation sy^ftems (A, 6}. . . , 

For almost all information systemsi some use of the system is required 
for example I it piay bo the only way to process orders or to compute <he 
payroll* Howevei^, it is expected that the nise of systems beyond basic 
input and transactions processing is voluntary, tot examplei a user 
generally decides ;voluntarily to perform an extensive analysis of an oul^put 
report instead of ;^g8t glancing, at it« - Attitudes have an^ctlon. component 
and favorable attitudes are consistent with high levels of voluntary use 
of information aysti^ms {4/6]. On the .other han$l| in the case of forced 
use of a: poor quality system^ user experiences niay actually lead to un^ 
favorable attitudes. \ ' 

THE STUDY \ ' . ^ 

\ 

Background \ 

The data in this |tudy were collected in a major university* the 
sample included administrative users of 26 separately identifiable 
batch computer systems, these systems ara concerned with accountlngi 
student record keeping* purchasing* fund raising, alumni record keeping, 
«etc. The administrative computer staff has qIso developed an online sys- 
tern for university processing .%nd several applications had been conver- 
ted to this system at the titne uf this study. Users are able 



to process certain employee data Itnd student records online. The online . 
system features special purpose programs along with a general Inquiry 
facility. It is felt that the nature of the batch and online applications 
and the tasks of the university admlnlatration create an environment for 
information systems which is comparable to other private^and public sector 
organizations. 

Research Design 

User attitudes toward information services department activities 
and perceptions of the quality of information systems service were measured 
by a questionnaire which was returned by U7 users on the university admin** 
istrative staff. (Allowing for terminations and questionnaires sent to 
non^users, the number of questionnaires returned was alm&jit 70% of those 
distributed.) This questionnaire Is similar in structure to those used in 
PciHt sludlouy the mujorily of responses have a range from one to seven |41. 

An independent measure of service quality is difficult to obtain. 
A panel of experts could be employed to rate existing systems; however » it 
is difficult to convene such a panel and an extensive effort would be 
required to perform the evaluation. To obtain a rating of systems quality 
independently from user perceptions in this study a structured question--^ 
naire was administered to Information services department staff members in 
operations and systems design positions « This questionnaire dealt with the 
quality of systems as seen by users; there were no questions on the tech- 
nical elenance, programming aspects or file structures of the systems. 

?^The questionnaire developed for this and prior studies Included a number 
of variables suggested by previous research ; see [1,2,4]. 



Between one and three 8t«££ mttb&ts In the information services department 
completed a questionnaire on ^ach of 'the 26 syatems and an attitmpt was 
made to|have ^ minimum of two people rate each system* Thes^ Responses 
were averaged to develop a single rating for each system which c^n vary from 
one to seven,' \ 

Factor analytic and correlation techniques were used on\oth the / 
user and the information services department questionnaire to form scales 
by averaging different Itoms on the questionnaire^ which were highly related. 
The scales on the user queationnalre are very similar to those of an earlier 
study t4]. User respondents were ajske^, to circle each of the 26 systems 
which they used on the questionnaire. 'To develop a single rating of the 
systema by the information services .department .staff, for each respondent/ 
the information services department Ratings for each of .the systems a user 
reported using were avetaged together. Thus, the independent ratings of 
service quality represent averages of the information syatems department 
system evaluations for all the^syatems with which a user r^isported having 
contact* 

VARIABLES 

The variables in the study are shown in Table li The variables 
labeled ''ISD Ratings" were derived from the process described above from 
the questionnaire administered to the information services staff. All 
other variables' were developed from the user questionnaire • In general^ 
all variables are coded so that a higher numeric response is more favorable. 



User Attitude* 



User Perceptions 



ISO Ratings 



Situational 



Aj Computer Potential 

Ag Attttudei Toward ISD Staff ^ 

Pj Input Quality 

Pg Output Quality 

P3 Rating of Online system 

P4 Training 

P3 , Contact with ISD 

Pg Management Support 

P7 Suitability of numbers dI: reports received 

Pg Would like more summary and exception reports 

Pj ' Invo^ement / 



Rj 4 Input Quality 

Output Quality ^ 
R3 Overall Quality " ^ ^ 

R^ Integration and Changes requested 
R5 System Usefulness 

Rg^ Documentation, Training, Accuracy 

R^ Online Performanjcp 



Sj Posltldn 



S2 Time on job 



TABLE 1 - 
Varlsbles in the Study 
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Information Services Department 



Education 



I2 

Uj^ Batch ayatem amount of uae 

Batch use for various activities/decisions 
Online system use 'of specific features 

U, Online system use In general 



/ 



TA]b1e 1 (continued) 
t 



^irlables in the Stufjy. v. 



\ 



\ 



Uaer Reactlona ' 

User. reactions to tnformatlon systema were measured by two attltu- ' 
dlnal variables In the study; coaputer potential aiid attitudes toward the 
•information services staff. The computer potential scale, conelsts of two 
questions In which the respondent rated, the clerical and managerial poten- 
tlal of computers for admirii4tratlve usS^, Attitudes toward the informatldn 
sei^lcea department staff include several items from the questionnaire 
describing staff competence, IntereU m the user, appreciation for the 
user's problems, otc,- ■ • 



User Perceptions ' 

^ A number of user perceptions of informab/on aervlces activities were 
included ln\hl8 study. Ratings of Input aad Output quality were developed 
from litems on the questionnaire describing the timeliness and accuracy of " 
Input -aitd output along with the general usefulness ^of output. Railn^.of 
the online system Include an evaluation of Its response time, reliability/ 
and ease of use," ^ \ 

The training scale Includes Items describing the amount of training 
rieported fey the user and per^tlons of the adeiquacy of, the training in 
preparing for the use of a nev system. Contact with the Information ser- 
vices^department was measured by en Item on which the respondent Indicated . 
the amount of tlm6^ spent on Iti^matiolv systems. related activities during 
th6 prior week. . Perceived manage^nt support is a single item on the 
questionnaire reflecting the user's perception of the extent to which 
management supports more administrative computer uee within the university/S 

A hiSh response to the scale on th6 suitability of the numbor >eport^ 
rocuiveU Indicates that the user is satisfied with the prcfiontH-ttiTort s he 



' receives or vould like to receive, even more reports from the computer . bn 
thA other..h«nd e high score on the desire for summery and exception reports 
means that the^er would lik,e to receive more of this type of report while 
^ - a low score indl^cates that he iV relativialy satisfied with the preserit^^ ■ 
form of reports. ' . . ; . 

. '. • ■ ■ ' ' . ■ V ■ '•■ ' ■■ • > 

Information Services Ratin g s V ■ . 

Seven scaUs evaluating tiie quality of inforoatt^n sy 
developed .from the ques^opnelre completed by the information services * 
, departolent staff. Input aria output, quality ^ the clarity of 

input/output documents, input errbr ratings, and the usefulness, accuracy 
and timeliitess of output reports. The overall quality of systems scaU' ' ' 
consists of ratings of systems flexibility,' the lack of pressure to repro, 
gram the system, and the completeness of the functiohs contained in the 
- sysi^em. ^ ^. ; - ■ ■■ . 

Aq analysis of the questionnaire indicated that a question on the S 
integration of systems and a lack o^ Change Request should be coinl?ined into 
a single rating 'scale (Variably R^) . Systems usefulness consists , of a * 
rating of how rtf^^ch the system can be used for 'decision making, trend analy- . 
sie, planjilng, taking action and problem finding. Documentation', training, ' 
^d systems accuracy also were found t^o form a single scale (Variable KA. 



srm a Single 
>le conAstf 



Finally, for online systems the rating scale c^jjAsts of items reflecting 
the rosponse^tttoe and stability of the online ftyatem* 

Sttiuational and Personal Factota " ^ * i 

• .Several sttuationar arid personal variables were developed! from the. 
usA questionnaire. Position' refers to the level of the respondent 'in ijho 



oteanlaatlon krid rangea^ f^^^^ top managew^t; Tl^ on 

the Job repreaeiitfl the lepgtft o£ time that ^he respondent has been in hie 
.present poBltlon. Age and education were also obtained frotn the question- 
naire ♦ ' ' ■ ' 

Use of Systems ;\[ . / . , • 

For batch systems, two scales were" derived from the us^er question! . 
nalre for the amount of ^seof-tnformat ion systems. The first variable (U^) 
deals' with the general use of systems, " The second variable ^U^) Is con- 
cerned with the, use of the flystefas .for ^specific functions like decision 
making, jirend analysis, etc. v V >^ 

Two usage scales .we.re also developed for online systems from the 
questl-onnalre. The iUBt~oi%,e-i^^U^6o^th^ with the- frequency 
of use of the online system, and thfi use of W<siflc functions Uke ttc \ 
Inquiry language. The second usagp scale (U^) represents use of th^.onllne 
system for different activities' like decision. making, trend analysis," etc.. 



RESULTS • M 

Because . of the multivariate nature of the hypotheses, step-wise 
multiple regression was^ used to analyze the^ata. The stop-wlfie'regression 
algorithm wag terminated when the coefficient of an Incoming variable wo61d 
be Insignificant at the .10 level. ^ < ' - 



2 ■ ' » ■ 

- A few of the coefficients are not significant at the .10 level due to 
changes m the equations which occured when subsequent variablos entered- 
onch variable was Significant at the .10 level or better when it entered' 
; the equation. - , . 



Syater^fl Quality and User fiQaottoniB 

The first hypothesis la that user reactions are positively associated 
with user perceptions of systems quality and Independent ratings of systems 
quality. The results for cotnputer potential are shown in Equation (I) and 

(1) Aj'- l'.59 + .€4 + .40 P5+ .12 + ,49 P^ + .19 in^ + .05 

, ,t (1.96) (2.32) (1.29) (1.99) (1.80) (1.68) 

attitudes toward the information services' staff in Equation (2)f These 

( (2) A2 - -1.05 +. ,01; Pj + .18 Pg V .^2 + .21 R3 

v. . (2.^7) * (.18) (1*68) , (1,38) 

two eqiiations.provide strong support for the first hypothesis, Per^eptioiis 
of tralnihg, contact with the in format ton services department and the suit- 
ability Of the number of reports received |ire positive predictors of . ratings 
of computer potential. Output quality rated bv the Ihforfflatlon services 
departnvent ataff Independently of users is also a positive predictor of' user 
ratings of computer potential. , 

Attitudes toward the information services staff' are positively asso- 
dated with. perceived output quality and managenent support in Equation (2). 
Independent information services department (ratings 6f Input quality and 
overall systems quality are also positive predictors of Tuser attitudes 
toward the Information services department staff. Thus, hypothesis 1 . 
receives substantial support from the'result^ In Equations (1) and (2)'. ' 




.n 


R 


*2 
R 


80 


.51 


,26 


77 ^ 


.54 . 


.20 


80 


.56 


.32 


79 


.43 


.19 


37 


.86 


.74 


36 


.82 


.67 



F 


P» 


4.34 


.001 


7. hi 


.001 


'^.65 


,001 


3.75 


.01 


17.44 


.001 


8.06 


,001 
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•TABLE a 
AtttibuUd Of Equations 



Ug6 of Systems ^ \ 

The 'second hypothesis la that favorable user attitudes and percep- 
tions are associated with voluntary use of Information systems. For the 
most part the use of the output batch systems Is voluntary, tii^ugh the 
ptovlslon of Input Is required. The majority of users, however, receive 
output reports from these systems as their major benefit. The results for 
batch use shown in Equations (3) and (4) generally support Hypothesis 2. 

(3) Uj^- 4.80 + .2lAj + .45 - .20 Pg - .46 + .50 R3 - .18 Ij 
- ' . . ■ <1.82) (1.78) (1,56) (i:88) (2.78) (2.51) 

(4) - 1.07 + .49 py + .56 + .10 ' 

■ (1.34) (2.58) (2.17) 

Computer potential is a positive piedictor of us^ In Equation (I) as Is the 
suitability of the number of reports received. The negative sign for the 
variable Pg can also be interpreted .aa favotable since this means that the 
user is satisfied with the format of present reports and does not feel 
the need for more summary or exception reports. The independent informa- 
tion services department rating of systems usefulness is positively 
associated with use,vonly the independent rating of input quality is 
negatively related to higher usage in Equation (3) . Given the fact that 
input is the least voluntary aspect of the use of an Inforp^tlon system, 
this finding is not surprising* In Equation (4) the Suitability of the 
number of reports received and ratings of overall systems quality are 
positively associated with systems u^e. 
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A PROBLEM ONLINE SYSTEM ^ 
Syatem Statua " ' ; 

major on Une syatem opey^ted by tha Information se 
ment had been under clevelopraent for aeveral yeara and was not completely ; 
Implemented at the tW of the study. There had been problems 
with machine failures and the majority of the usets represented clerical 
personnel who had tc use the system to enter data; the old alternative of 
batch processing wa^ no longer available . . the system Is riot strictly 
online since somfe bajtch reports are produced from the data collected on- 
line, ;. - ..J;" /: ^ - ^''.^^ 

As Shown in Table 3 the perceptions of users of the dnllne system 
are much less favoralllc than fc^ users of the batch system, (It should, 
be noted that these ratings also Include any batch systeiM used by the respon- 
dent). Input/output (juallty is rated slgnlflcanajTl^sWer for ail systems 
by users of the online > system. This finding represents a contrast tb a paat 
study In which online sifstems were associated with, more favorable ratings 
of input and outpi/tx|4).\ More Inforpiatlon services department contact 
and lnv.olvement are reported by users of the online system, probably 
be<^ause Of the systeu's recent development. Hpwever,: less management 
.support is perceived by thes'e usefr, The only fsvorable response Is the 
ariable which represents greater satlsfactloh'Wlth the current nuinber of ' - 
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Means 








Variable - 


Davcn ^n"oD ) . 


Online (n*»41) 


t Value ' 




Pi 


. • J. J 




3.29 


".001 


■2 






2.50 


.02 


5 




l«6o 1 


-1.86 


.07 


6 




4^73 I 


3.02 


.004 


P- 




2,46 1 


-3.72 


.001 


P-^ 




2.76 .'■ 

. " V ' i 


-2.17 


, .03 


P^ 
9 


1 . '07 ■■■■ ■■■■./■■- 
1 •7/ 


2.96 


-4.53 


.001 


2 




1 

'4.97 I 


6.06 


.001 


3 






^ 1.97 , 


.05 


' R, 




c OA 


-3.30 


.002 


■-■.'5' - ■■ : 




^ *f<**l 


3.77 


,001 






y • 


4.91 


.001 




■ 5U2. 


2,41 


5.30 


" .001 


h 




5,23 


•3.90 


.001 




43.48 


34\50 


4.12 


.001 



^f:!;;^;i.■:^^^-^^■■■x■;^^.; .:. 



TABLE '3 

Online vii Batch System Users 
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Output quiiltty «nd' overall quality are rated higher for exclusively batch 
userfi. The general us^s t^t the ayatcn; and documentation and accuracy 
ratings follow thp same pattern. The information services department staff 
ratings for the group using the online system is , Acre favorable than 
exclualvely batch system users for the amount of system Integration and 
the changes needed I ; , . 

Personal characteristics alsc differ significantly among the res- 
pondent3 who useyonly batcher systeftis and those who also use the online sys- 
tem* Use of the online system Is aspoclated with less tlM on the Job and 
with youiiger and more educ'sted employees. Such a group may be the hardest 

' ' / ^ ' ' ' ' V.) ' ' ' r 

to satisfy with a computer system as they may expect more and be more 
impatient With problems •/ The. relatively poor ratings of the online system 
are probi^bly due both to/ these chsriictdristics and the problems encountered 

In evaluating a .recently designed and implemented system/ The online sys- 

■ .- ■ . , j • ■ • - 

tern. Is compared with batch systems which > In general^ have been In smooth 
operation for a much ^nger period. 



Use of the Online System 

Given these problems with the online system/ ate there significant 
differences In the factors which ..ate associated with the use, of th^ system? 

■ ' • " ^ ^ - I ' ' 

The results In Equations (5) and (6) indicate that even for the'' relatively 
poorly rated online system, uie Is positively associated ^ith user bercep- 
tion of^ systems quality a<td/informf tlon services if^fclngS'bf systems 
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(6) » -5.42 + .52 .70 +, .8$,?^ - 2.40 R^ + 1.04 R^ 4 1.48 R^ + .1 M \f 

(2.87) (3.68) (4.98) (3.99) (3.20) (2.96) (2.05) 

•»'^-'" i ■ . • ' ■ " ■ ■ 



performance. Use of the systett^ 1$ also =po8ttlVely related to Information 
services department ratings of overall quality arid systems usefulness. , 
However, use of the system is negatively related to Information' services ■ 
department ratings of output quality and documentation, training, and 
accuracy. For general- use in Equation (6) ratings of computer potential 
by users Is a positive predictor of use. However in both Equations (5) 
and (6), attitudes toward the Information services staff are negatively 
associated with use. This result is contrary to t^je geheral findings for 
batch users in Equations (3) and (4),- though \ understandable 
given the unfavorable ratings of the system and its Vplementation prob^ . 
leraf . Users of the online syste^j have h^a high Uvell of - cotjtact with the 
information services department staff under the difficult circumstances 
surrounding conversion and implementation so unfavorabl^ staff ratings 
might be expected. 

E x tonstons 

- - ,• \i ■ , ■ , >i ' 

The research techniques and instruments in this study can be used 

by any organization to evaluate Its Information systems. This study has 

demonstrated an association .between systems quality and user reactions to 

information systems. The survey, instruments could eaaily be admlnist^ed 

and analyzed to provide management with Information on user reaction to 

specific systems In an organization; Both the systematic statistical 

results and the suggestions from open-ended sections of the questionnaire • 

cahiie extremely valuable to management. For example-the sample- can be ' 

pnrtlttoned by systems and ^ppUeatlohs which differ from the mean rating 
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base of similar studies becjimes available through this and other research, 
norms < ->uld be developed for different systems and.diffe.rent.<types of/ ' 
industries with which. an organlgatton could compare its own, systems 
evaluation results • - * . | , - . 

DISCUSSION . .* 

The two hypotheses received substantial support from this study. 
The quality of systems rated independently and as perceived by users is 
essociated with favorable us^r attitudes. Favorable attitudes and ratings 
oif systems quality are also associated with . the une of systems.. However, 
these results have not demonstrated causality due to the cross-sectional 
nature of the research. It does seem reasonable., given the independent 
evaluation of systems qualityj that quality influences attitudes. It is hard 
to believe thtit user attitudes could lead directly to better systems qual- 
ity as rated independently by the infortnation services department. How- 
ever, the direction of any possible relationship between attitudes and the 
use' of the system not completely clear. Voluntary use is expected to 
be predicted by favorable attitudes. On the other hand, If use of a system 
Is mandatory, the use of a poor quality system could lead to the develop- 
ment of unfavorable user attitudes. 

Froi^ J practical standpoliit, the quality 6f systems is more undev ' 
■the control 6f the informatl6n ser^jLce? de^aiftment staff Ithan are user atti- 
tudes, the findings of thls^^tudy suggest th^^ Wf6re iiVidertdVin^ new 
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best predictor of cthe benefits of a new syatem, and succesafullyvfunctlonlng 
systems are a prerequisite to obtaining user co6p<iratlon In systems design. 

During the development of any system, the information services 
depdrttr»ent st>«»ff should consider user attitudes and perceptions and should ' 
try to measure the quality of systems from the user's standpoint. For an 
example of a systems design activity Including conscious consideration of 
user reactions, iee (3J. 

Finally, If . the' Information services department and management, |ive 
faith that Information systeirfo are , worthwhile and contribute to the oirganl- 
zatlon, then they should try to jencourage high levels of systems use. 
Though causajlty has not b6en demonstrated, £ri^>rl 'arguments and the data- 
support the goals pf developing favorable user attitudes and high" quality 
systems to encourage the use of information systems. 
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